Portable Devices as Videoconferencing Peripherals

ABSTRACT

A videoconferencing system has a videoconferencing unit that use portable devices as peripherals for the system. The portable devices obtain near-end audio and send the audio to the videoconferencing unit via a wireless connection. In turn, the videoconferencing unit sends the near-end audio from the loudest portable device along with near-end video to the far-end. The portable devices can control the videoconferencing unit and can initially establish the videoconference by connecting with the far-end and then transferring operations to the videoconferencing unit. To deal with acoustic coupling between the unit&#39;s loudspeaker and the portable device&#39;s microphone, the unit uses an echo canceller that is compensated for differences in the clocks used in the A/D and D/A converters of the loudspeaker and microphone.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is filed concurrently with U.S. application Ser. No.13/282,609 (199-0890US1), entitled “Pairing Devices in Conference UsingUltrasonic Beacon” by Peter L. Chu and Yibo Liu; and U.S. applicationSer. No. 13/282,633 (199-0890US2) entitled “Compensating for DifferentAudio Clocks Between Devices Using Ultrasonic Beacon” by Peter L. Chuand Yibo Liu, both of which are incorporated herein by reference intheir entireties.

BACKGROUND

Videoconferencing units are complex devices, requiring complicatedfunctions to operate. Therefore, videoconferencing vendors createspecial remote controls and menus to operate the units, but these can behard to learn and use. This is especially true when the unit has severalfunctional capabilities.

Additionally, good quality microphone pickup in a videoconferencerequires the participants to be no more than 3-feet from themicrophones. Therefore, videoconferencing vendors provide specialmicrophones for placement on tables near the participants in aconference room. The microphones often have wires that can get in theway. As a solution, wireless microphones are available forvideoconferencing systems, but they need to be repeatedly recharged ifused often, which can be inconvenient.

The subject matter of the present disclosure is directed to overcoming,or at least reducing the effects of, one or more of the problems setforth above.

SUMMARY OF THE DISCLOSURE

In a conferencing system, participants use portable devices asperipherals of a conferencing unit. During the conference, for example,the portable devices communicate audio to the conferencing unit in awireless connection, while the conferencing unit communicates with afar-end in a conferencing connection. With the system arranged in theseconnections, each participant can use the microphone on his or herportable device as a personal microphone, which will typically be within3-ft of the participant. Additionally, the participants can use anapplication running on their portable devices throughout the conferenceto control aspects of the conferencing unit.

As noted above, the portable device can be used in a conference, whichcan simply be an audio conference. Alternatively, the conference can bea videoconference involving video and audio. When video is involved, theportable device can optionally communicate video in addition to orinstead of audio. For its part, the unit can be a videoconferencing unitcapable of handling both video and audio. Accordingly, the teachings ofthe present disclosure as properly understood can apply to audioconferencing and videoconferencing. Throughout the disclosure, however,reference is made to videoconferencing, although the disclosed teachingsshould be understood to apply to just as well to audio conferencing.

To conduct the videoconference, for example, the unit receives near-endaudio obtained with microphones of the portable devices via the wirelessconnections. The unit also receives audio from the unit's microphone(s),which can be table-mounted microphones, a pod, or the like. At the sametime, the unit receives near-end video obtained with one or morecameras. Depending on the implementation, the one or more cameras canbelong only to the videoconferencing unit, but video from cameras on theportable devices can also be used. The unit then communicates thenear-end audio and the near-end video to the far-end via thevideoconferencing connection.

As the videoconference is conducted, various participants speak, and theunit switches which near-end audio (and optionally which near-end video)source that it outputs to the far-end. For example, if a participanthaving a portable device speaks, the unit can output the near-end audioobtained with that device's microphone. When selecting the audio fromthe portable device, the unit can also output the video obtained withthe portable device's camera.

Because several microphones may be sending audio to the unit, the unitpreferably selects the near-end audio associated with the microphonehaving the greatest level, which can be the microphone of a portabledevice or the unit. Because the audio has the greatest level, it is morelikely associated with the current speaker during the conference.

Before the videoconferencing unit communicates with the far-end, aparticipant can initiate the videoconference by establishing thevideoconferencing connection to the far-end using his or her portabledevice independent of the unit. In this initial arrangement, theportable device can receive far-end video and audio from the far-end foroutput with a display and loudspeaker on the portable device. Likewise,the portable device can similarly send near-end video and audio obtainedwith the camera and the microphone of the portable device to thefar-end.

Later, the participant at the portable device can then transfer thevideoconferencing connection with the far-end from the portable deviceto the videoconferencing unit. After transfer, the unit can take overreceiving far-end video and audio from the far-end, sending the far-endvideo to the unit's display, and sending the far-end audio to the unit'sloudspeaker. Yet, the videoconferencing unit can still use the portabledevice's microphone for audio capture and the device's camera for videocapture at the near-end depending on the circumstances.

Rather than starting the videoconference, the portable device can joinan existing videoconference being conducted by the unit. Preferably, thedevice can automatically pair with the unit by obtaining the unit's IPaddress encoded in an acoustic or ultrasonic beacon output by the unit'sloudspeaker. Using the decoded IP address, the device establishes awireless connection as a peripheral with the unit for sending audio andvideo for the videoconference.

As the unit receives far-end audio from the far-end and outputs it to aloudspeaker, the outputted audio may be acoustically coupled with themicrophone of the portable device. To deal with this, an echo cancellerof the unit cancels output audio from the loudspeaker that is present inthe near-end audio obtained with the portable device's microphone.

Internally, the portable device's microphone uses an analog-to-digitalconverter that functions with a first clock, while the unit'sloudspeaker uses a digital-to-analog converter that functions with asecond clock. Thus, these two clocks may operation with differentfrequencies, which can reduce the effectiveness of the echo canceller.To handle the mismatch in the clocks, a clock compensator of the unitand/or the portable device compensates for a frequency differencebetween the clocks.

The foregoing summary is not intended to summarize each potentialembodiment or every aspect of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a videoconferencing system according to the presentdisclosure.

FIG. 2 schematically illustrates a portable device for use with thedisclosed system.

FIG. 3 shows a videoconferencing interface for the portable device.

FIG. 4A illustrates a process for conducting a videoconference with thesystem of FIG. 1 when the portable device initiates the videoconference.

FIG. 4B illustrates a process for conducting a videoconference with thesystem of FIG. 1 when the portable device joins the videoconference inprogress.

FIG. 5 schematically illustrates an echo-canceller for thevideoconferencing unit and clocks for the videoconferencing unit andportable device of the system.

DETAILED DESCRIPTION

A. Videoconferencing System

A videoconferencing system 10 according to the present disclosureillustrated in FIG. 1 has a videoconferencing unit 100 with aloudspeaker 122 coupled to an audio interface 120. As is typical, theunit 100 can use one or more table-mounted microphones 124, a microphonepod, ceiling microphones, a microphone array, or other acoustic devicescoupled to the audio interface 120 for capturing audio, but suchacoustic devices may be optional in the present system 10. The system 10also has a display 142 and one or more cameras 144 coupled to a videointerface 142. For example, the system 10 can have two cameras 144—onefor capturing a wide angle view of the videoconference environment andone for capturing tight views of participants.

Although reference is made herein to the system 10 being used forvideoconferencing, the teachings of the present disclosure can applyequally to audio conferencing lacking video. Nevertheless, referencewill continue to be made here to videoconferencing for the sake ofdescription.

Internally, the unit 100 has a processing unit 110 with an audio codec112 and a video codec 114 respectively connected to the audio and videointerfaces 120 and 140 for encoding and decoding audio and video for thevideoconference. Finally, the unit 100 has a network interface 130connected to the codecs 112 and 114 for communicating audio and videobetween the near-end unit 100 and far-end units 30.

During a videoconference, many of the participants would likely havetheir own portable device 50 available because portable devices 50 havebecome a common appliance. Users are comfortable using the portabledevices 50, and the videoconferencing unit 100 can use the portabledevices 50 as videoconferencing peripherals as disclosed herein.

In general, the portable devices 50 can include any of a number ofavailable devices, including, but not limited to, peripheral devices,cellular telephones, smartphones, tablet PCs, touch screen PCs, PDAs,hand-held computers, netbook computers, laptop computers, and the like.Additionally and as discussed later, the portable devices 50 can haveprocessing capabilities and functionality for operating a camera, adisplay, and a microphone and for connecting to a network, such as Wi-Finetwork, Internet, and the like.

In general, the network interface 130 can connect to the far-end units30 via an Ethernet connection, a wireless connection, an Internetconnection, a POTS connection, or any other suitable connection forvideoconferencing or combination thereof. As part of the networkinterface 130 or separate therefrom, the unit 100 includes a peripheralinterface 150 that enables the videoconferencing unit 100 to communicatewith local peripherals, such as the portable devices 50. In the currentexample, the network interface 130 connects the unit 100 to a localintranet of a local area network (LAN) 132, which connects in turn to awide area network (WAN) 136, such as the Internet. The LAN 132 may havea wireless local area network (WLAN), Wireless Fidelity (Wi-Fi) network,personal area network (PAN) (e.g., Bluetooth), or similar type ofwireless network 134 for connecting to the portable devices 50.Accordingly, participants can connect their portable devices 50 with thewireless network 134 to the LAN 132 so transport between the portabledevices 50 and the videoconferencing unit 100 can use the wirelessnetwork 134 and the LAN 132.

In many instances, the portable devices 50 can have high qualitymicrophones 74, and the unit 100 can use the device's microphones 74 asvideoconferencing microphones. In this way, several of the participantscan use the microphones 74 on their devices 50 as personalvideoconference microphones, and the close proximity of each microphone74 to each participant will likely offer high quality audio pickup forthe videoconference. Additionally, the portable devices 50 can have highquality cameras 84, and the unit 100 can use the devices' cameras 84 aspersonal videoconference cameras in close proximity to the participants.

B. Portable Device

Looking at a portable device 50 in more detail, FIG. 2 schematicallyillustrates an example of a portable device 50 for use with thedisclosed system 10. For audio, the device 50 has a loudspeaker 72 and amicrophone 74 coupled to an audio interface 70, while the device 50 hasa display 82 and a camera 84 coupled to a video interface 80 for video.Internally, the portable device 50 has a processing unit 60 with anaudio codec 62 and a video codec 64 respectively connected to the audioand video interfaces 70 and 80 for encoding and decoding the audio andvideo. Finally, the portable device 50 has a network interface 90connected to the codecs 70 and 80 for communicating audio and video withthe near-end unit (100) and far-end units (30).

In one type of connection, for example, the network interface 90 canconnect to a typical cellular network 92 if the device 50 can be usedfor cellular communications. In another type of connection, the networkinterface 90 can connect to the LAN 132 via the wireless network 134 sothe portable device 50 can communicate with the videoconferencing unit(100) or far-end (30). As one skilled in the art will appreciate, anyother type of connection can be used to communicate between the portabledevice 50 and the videoconferencing unit (100). As will also beappreciated, establishing a connection between the portable device 50and the videoconferencing unit (100) and far-end (30) requiresparticular protocols, applications, accounts, and other details that arepre-arranged for the connection to be possible so the details areomitted here.

C. User Interface

To operate in conjunction with the videoconferencing unit (100) andoptionally controlling the unit's functions, the portable device'sprocessing unit 60 has a videoconferencing application 66 with a userinterface. When operated, the application 66 allows a participant to usethe portable device 50 as a peripheral to the videoconferencing system(10). FIG. 3 shows an example videoconferencing interface 67 for theportable device's application (66). Participants can use the portabledevice's interface 67 to operate in the videoconference system (10).

As shown in the example of FIG. 3, the user interface 67 has a number ofuser interface items available for the application (66). These items canbe individually configured by the user, although some of them mayoperate automatically by default. The items can include, but are notlimited to, starting a videoconference, ending a videoconference,joining a videoconference, sending content for the videoconference,using the device's microphone 74 and/or camera 84 for thevideoconference, transferring the videoconference from the portabledevice 50 to the unit 100, and performing additional functions. In oneadditional function, for example, the participant can indicate a desireto be queued to ask a question in the conference and to be made activespeaker when permitted. Some of other additional functions can besimilar to the typical functions available on a conventional remotecontrol of a videoconferencing unit, such as controlling loudspeakervolume, moving cameras, changing display options, etc.

Some general discussion of the user interface items follows. Byselecting to start a videoconference, for example, the portable device50 can be used to initiate a videoconference as discussed herein. Byselecting to join a current videoconference, the portable device 50 canbecome a peripheral device to the videoconferencing unit 100 currentlyconducting the videoconference. By selecting to use the device'smicrophone, camera, or display, the user can configure how the portabledevice 50 is to be used as a peripheral to the videoconferencing unit100.

Finally, by selecting to send content, the user interface can allow theparticipant to send content from the portable device 50 to thevideoconferencing unit 100 for incorporation into the videoconference.The content can include a video, an image, a document, a presentation,etc. that resides on the portable device 50. Thus, in this arrangement,the portable device 50 can act as a content or presentation device forthe unit 100 and can have a video player, an image application, adocument application, a presentation application, etc. for generatingthe content so the device 50 can send the content to the unit 100through the network connection.

D. Videoconference Process

Having an understanding of the videoconferencing system 10, thevideoconferencing unit 100, and the portable device 50, discussion nowturns to how the portable devices 50 can be used with thevideoconferencing unit 100 during a videoconference. As briefly hintedabove, a participant can start a videoconference with a portable device50 and can later transfer it to the videoconference unit 100. FIG. 4Aillustrates a process 200 for conducting a videoconference with thesystem 10 of FIG. 1 (to which reference is concurrently made) when theportable device 50 initiates the videoconference.

To initiate the videoconference, a participant connects to the far-end30 using their portable device 50 and the videoconferencing application66 (Block 202). Any of the various network connections can be used toestablish the videoconference. For example, the portable device 50 canconnect via the Wi-Fi network 134 to the LAN 132 and then to the WAN(i.e., Internet) 136. If the portable device 50 has cellularfunctionality, then the videoconference can be established with acellular telephone call as known in the art.

Once connected to the far-end 30 (Block 204), the portable device 50receives and renders the far-end video and audio for output on theportable device 50 using its display 82 and loudspeaker 72 (Block 206).At this point, the portable device's existing camera 84 and microphone74 can transmit video and audio to the far-end 30 as part of thevideoconference (Block 208).

When desired, the participant selects to transfer the videoconference tothe near-end videoconferencing unit 100 (Block 210). To transfer thevideoconference, for example, the participant initiates the transfer byselecting an interface item on the device's conferencing application(66). The portable device 50 transfers the videoconference connectiondetails to the videoconferencing unit 100, and the videoconferencingunit 100 in response establishes its own videoconference connection withthe far-end 30.

For example, the conferencing application 66 on the portable device 50can have a “transfer” button for the participant to select on the userinterface (67). (See FIG. 3.) When the transfer is selected, the userinterface (67) can have a number of input screens for the participant toenter the IP address or other identifier of the far-end 30 so thevideoconferencing unit 100 can establish the connection. Other detailsmay also be entered in the user interface (67), such as passwords,connection information, participant tags, etc.

Once the transfer is completed and verified, the portable device 50disables its connection with the far-end 30 so it will only use itslocal connection to the videoconferencing unit 100 for thevideoconference. At this point, the unit 100 begins rendering thefar-end video and audio with the higher quality processing availablewith the unit 100 (Block 212).

Now as the conference proceeds, near-end video processing switches fromthe portable device's camera 84 to the unit's higher quality one or morecameras 144, and the audio processing also switches from the device'smicrophone 74 to the unit's microphone 124 (Block 214). The unit's oneor more cameras 144 can use the unit's microphones 124 (and morespecifically a microphone array (not shown)) to determine locations ofparticipants when speaking using well-known techniques for directingcameras at sources of audio during a videoconference. Yet, as notedherein, the portable device 50 remains paired in communication with thevideoconferencing unit 100 via the network connection (e.g., Wi-Finetwork and LAN) so the device 50 can be used for video and audioprocessing when chosen to do so. In other words, the device's microphone74 can still be used to obtain the videoconference audio, and thedevice's camera 72 can be used to obtain videoconference video.

At the outset, however, audio and video processing may be handled by thevideoconferencing unit 100 as shown in FIG. 4A. When the unit 100 isprocessing audio, the portable device's loudspeaker 72 can be muted sothat it does not output audio during the videoconference, or theportable device 50 may simply not receive audio for output from the unit100 via the local network connection.

At some point during the videoconference, the participant with thepaired device 50 may speak, and the unit 100 and/or the device 50monitors whether audio from the device's microphone 74 is obtained(Decision 216). In one implementation, the paired device 50 obtainsaudio with its microphone 74 and transmits this audio in its pairednetwork connection to the videoconferencing unit 100 regardless ofwhether the participant with the paired device 50 is speaking. In turn,the videoconferencing unit 100 determines whether the microphone 74 iscapturing active speaking by the participant. As an alternative, thepaired device 50 obtains audio with its microphone 74, but does not sendthe audio to the unit 100 unless the device 50 determines there isactive speaking.

In any event, if there is no device audio (either because none of thedevices 50 is using its microphone 74 for audio or because none of theactive device microphone 74 is capturing speech) (No-Decision 216), theunit 100 continues using its microphone 124 for audio capture (Block226). Following on this, the unit 100 may still continue using itscamera 122 for video capture as there may be no need to switch to videocapture from a portable device 50 when a participant is not speaking(Block 228).

At some point in the videoconference, however, one of the devices 50using its microphone 74 for audio pickup does obtain audio from itsparticipant, and the unit 100 uses this audio for the videoconference asdetailed below (Yes-Decision 216). In using the audio, thevideoconferencing unit's audio codec 112 encode the microphone's audioas conference audio and sends the conference audio from the portabledevice 50 to the far-end 30. If several device microphones 74 (andpossibly the unit's microphone 124) are concurrently capturing audio,then the unit 100 selects the microphone 74 or 124 having the loudestinput (Block 218) and uses that device's audio for the videoconferenceaudio (Block 220). This comparison may prevent the unit 100 fromchoosing a source of audio input in error. Any of the numeroustechniques can be used for determining the acoustic energies of inputaudio and comparing them to find the strongest input audio.

If some of the participants do not have a portable device 50, the unit100 can obtain conference audio from the existing microphone (124) (ifavailable) or from the closest portable device's microphone 74(presuming it is the loudest). In the end, the unit 100 does notnecessarily need to know how many participants are present and how manyhave a portable device 50 capturing audio, although this could be usefuland can be manually or automatically determined. At most, the unit 100needs merely to select a microphone (whether one of the portabledevice's microphones 74 or the unit's existing microphone 124) havingthe loudest captured audio for input.

The device 50 from which audio is used may also have its camera 84available for capturing video for the videoconference, and the unit 100decides whether the device's camera 84 can capture video of the speakingparticipant for the videoconference (Decision 222). If not available(either because there is no camera 84 or it is not enabled), the unit100 continues using its camera 122 for the conference video (Block 228).Otherwise, the unit 100 uses the video from the device's camera 84 forthe videoconference (Block 224).

At any point during the process 200, the participant can elect to sendcontent from the portable device 50 to the videoconferencing unit 100for incorporation into the videoconference (Block 225). As noted above,the content can include a video, an image, a document, a presentation,etc. that resides on the portable device 50, and the portable device 50can send the content through the network connection (e.g., wirelessnetwork 134 to the LAN 132) to the unit 100. In turn, the unit 100 canincorporate the content into the data sent to the far-end as part of thevideoconference.

Depending on the capabilities of the portable device 50 and the type ofcontent involved, the content can be sent as a data stream or in a datafile to the unit 100. Additionally, the portable device 50 acting as acontent or presentation device may or may not be capable of acting as aperipheral (i.e., capturing and sending audio and/or video from thedevice 50 to the unit 100), as this would depend on the processingcapabilities of the device 50. All the same, the portable device 50 canbe configured to send one or more of audio, video, and content to theunit 100 at any given time during the videoconference as configured bythe participant.

The current processing arrangement may continue as long as thecircumstances remain the same (Decision 230). At some point, theparticipant with the device 50 delivering audio (and optionally video)may stop speaking, the participant may disable the paired connection ofthe device 50 with the unit 100, the participant may leave the room withthe device 50, or any other type of change may occur. Accordingly, theunit 100 will stop its processing arrangement and will return again todetermining whether to use audio and video from a portable device 50 orfrom the videoconferencing unit 100 (Decision 216).

The scenario above has discussed how the portable device 50 can initiatethe videoconference with the far-end 30 and then transfer it to thevideoconferencing unit 100. In an alternative scenario, thevideoconferencing unit 100 can initiate the videoconference and canbegin rendering audio and video obtained with the conventionalmicrophone(s) 124 and the unit's camera 144. Then, during thevideoconference, participants with portable devices 50 can connect orpair with the videoconference unit 100 to join the videoconference.

Once the device 50 pairs with the unit 100, the participant can use theuser interface application 66 to perform any of various functions, suchas display the videoconference video with the portable device's display82, reproduce the videoconference audio with the loudspeaker 72 or aheadphone output, start and stop the videoconference, etc. Morespecifically, the paired device 50 can be used as a microphone andoptional video peripheral during the videoconference as detailed herein.

To that end, FIG. 4B shows a process 250 for a portable device 50 tojoin a videoconference as a peripheral to the videoconferencing unit 100during a videoconference. Initially in the process 250 of FIG. 4B, thevideoconferencing unit 100 repeatedly sends an acoustic or ultrasonicbeacon encoding its IP address into the surrounding environment (i.e.,conference room, lecture hall, etc.), as the unit 100 conducts thevideoconference (Block 252). A participant in the room having a portabledevice 50 running a conferencing application (66) can select to join thevideoconference conducted by the unit 100, and the device's application(66) decodes the IP address from the acoustic or ultrasonic beacon(Block 254). Using the decoded IP address and wireless network (134) andLAN (132), the device 50 sends a response to the unit 100 to join thevideoconference (Block 256), and the device 50 and unit 100 exchange ahandshake to initiate the communication between them so the device 50can be paired with the unit 100 (Block 258).

As the videoconference continues, the near-end unit 100 renders far-endaudio and video (Block 260) and obtains near-end video from the unit'scamera 144 and audio from the unit's microphone 124 (Block 262). Yet,when appropriate, the portable device's microphone 74 capturing audiocan be used for the videoconference, and the device's camera 84capturing video can also be used for the videoconference.

From here on, processing can continue as in the previous arrangement ofFIG. 4A. As long as none of the portable devices 50 capture audio withthe greatest level, then the unit 100 can continue using its microphone124 and camera 144 to send to the far-end 30 (Block 276-278).

At some point, for example, one of the devices 50 using its microphone74 for audio pickup obtains audio from its participant, and the unit 100uses this audio for the videoconference as detailed below (Yes-Decision266). If several device microphones 74 and 124 are obtaining audio, thenthe unit 100 selects the microphone 74 and 124 having the loudest input(Block 268) and uses that device's audio for the videoconference audioby sending the audio to the far-end 30 (Block 270).

When the portable device 50 used for audio has its camera available forcapturing video for the videoconference, the unit 100 decides whetherthe device's camera 82 can capture video for the videoconference(Decision 272). If not available, the unit 100 continues using itscamera 122 (Block 278). Otherwise, the unit 100 accepts the video fromthe device's camera 82 for the videoconference (Block 274). Finally, atany point during the process 250, the participant can elect to sendcontent from the portable device 50 to the videoconferencing unit 100for incorporation into the videoconference in the same manner describedpreviously (Block 275).

The current processing arrangement may continue as long as thecircumstances remain the same (Decision 280). At some point, theparticipant at the device delivering audio (and optionally video) maystop speaking, and the unit 100 will return to determining whether audioand video should be captured with another portable device 50 or with thevideoconferencing unit 100 (Decision 266).

The choice on whether to output video from a portable device 50 in theprevious processes 200, 250 relied on whether the portable device 50 wasbeing currently used to obtain input audio for the videoconference. Inother words, if the portable device 50 is not selected for audio, thenthe device 50 would not be used for obtaining video in the previousprocesses 200, 250. Yet, other arrangements could be used so that aportable device 50 obtaining only video and not audio could be used toobtain video for the videoconference without obtaining audio.

E. Automatic Pairing

As noted above, a portable device 50 can automatically connect or pairwith the videoconferencing unit 100 using acoustic or ultrasonic pairingso the portable device 50 and videoconferencing unit 100 can communicatewith one another through the network connection. To initiate thepairing, the videoconference unit 100 transmits an acoustic orultrasonic beacon encoding its IP address using its loudspeaker 122, andthis IP address can be the unit's address in the local intranet.

The acoustic or ultrasonic beacon is preferably beyond of the audiblerange but low enough so conventional loudspeaker and microphonecomponents can still have a useful signal response. Therefore, thefrequency is preferably above 20 kHz. One acceptable frequency is 21kHz. Since this frequency is above the human range of hearing, it cannotbe heard during the videoconference. Additionally, the beacon purposelyhas a low volume when output. As will be appreciated, the beacon can useany other frequency and need not be in the ultrasonic range.

Transmission of the IP address and any other pertinent information inthe beacon preferably uses an audio frequency-shift keying (AFSK) formof frequency modulation with the carrier signal of about 21 kHz. Asnoted above, the device 50 having a microphone 74 and running theperipheral conferencing application (66) detects the beacon, decodes theIP address based on the AFSK modulation, sends a response to thevideoconferencing unit 100 via the network connections, and the twodevices 50 and 100 share a handshake so the device 50 can be used in thevideoconference.

The pairing can disconnect automatically when the device 50 leaves aroom or is actively disconnected by the participant. The device 50 canthen connect automatically to another videoconferencing unit whenentering another room. Unlike Bluetooth technology, the acoustic orultrasound beacon can be limited to the environment (room) of thevideoconference's unit 100 so there may be less chance ofcross-interference between different videoconferencing units in abuilding causing incorrect pairing of devices 50 to the units 100. Thus,the ultrasonic pairing can allow the unit 100 to identify specificallythat the device 50 is in the same environment as the unit 100 and notlocated somewhere else in a building.

Although the unit 100 is described as sending an acoustic or ultrasonicbeacon encoding its IP address, each of the devices 50 can do the same.Therefore, in a reverse pairing arrangement, the unit 100 detects anacoustic or ultrasonic beacon from a portable device 50, decodes thedevice's IP address, and establishes the network connection with theportable device 50 based on the decoded IP address. Moreover, theportable devices 50 can pair with one another using their IP addressesencoded in ultrasonic beacons.

Moreover, when a device 50 sends an acoustic or ultrasonic beacon asdiscussed above, the videoconferencing unit 100 can uses its microphones124 (and specifically a microphone array) to determine the location ofthe device 50 in the videoconferencing environment. In this way, theunit's one or more cameras 144 can more readily direct its view toward aparticipant having emitting device 50 when that participant is speaking.This can be especially useful when the speaking participant ispositioned a distance from the units' microphone 124 or array andtraditional techniques for locating a source of audio may be limited.Ultimately, the device 50 can operate as an acoustic or ultrasonictransmitter so the videoconferencing system 10 can operate in a mannersimilar to the system disclosed in U.S. patent application Ser. No.11/872,303, filed 15 Oct. 2007 and entitled “Ultrasonic Camera TrackingSystem and Associated Methods,” which is incorporated herein byreference in its entirety.

F. Echo Cancellation

As noted above, the portable device's microphone 74 can capture audioand transmit it through the network interface 90, wireless network 134,and LAN 132 to the videoconferencing unit 100 for sending to the far-end30. At the same time during the videoconference, the videoconferencingunit's loudspeaker 122 outputs audio at the near-end for thevideoconference environment. Because the portable device's microphone 74and the unit's loudspeaker 122 are in the same environment, acousticcoupling can occur when the portable device's microphone 74 directlycaptures audio output by the loudspeaker 122. As is known, this acousticcoupling can cause an “echo” to be transmitted by the unit 100 to thefar-end 30, and the far-end 30 will hear its own audio returned back toit at a delay. Therefore, the videoconferencing system 10 preferablyattempts to reduce the detrimental effects of any acoustic coupling thatmay develop during the videoconference.

To that end, discussion turns to FIG. 5. As schematically shown andpreviously discussed, the videoconferencing unit 100 receives outputaudio from the far-end 30 via the network interface 130, and the audiocodec's decoder 112a decodes the output audio for output by the unit'sloudspeaker 122. To output the audio, a digital-to-analog converter 119uses a sample rate conversion based on a clock 118 and converts thedigital output to analog output, which the loudspeaker 122 then outputsto the environment.

In turn, the portable device's microphone 74 captures audio in theenvironment. In capturing the audio, the portable device'sanalog-to-digital converter 69 uses a sample rate conversion based on aclock 68 and converts the analog input to digital input. Some of theaudio captured with the microphone 74 may be acoustically coupled fromthe loudspeaker 122. Nevertheless, the portable device 50 communicatesthe captured audio to the videoconferencing unit 100 via the networkconnection (e.g., Wi-Fi network, LAN, and the like). In turn, the unit'saudio coder 112b encodes the input audio for sending to the far-end 30.If the audio were left alone, any acoustically coupled audio from theloudspeaker 122 to the microphone 74 would be sent via the networkinterface 130 to the far-end 30 as an “echo.”

To deal with acoustic coupling and the resulting echo, thevideoconferencing unit 100 has an echo canceller 115. Using any of theknown echo cancellation techniques, the echo canceller 115 compares theinput audio from the microphone 74 with the output audio for theloudspeaker 122 and attempts to remove the output audio from theloudspeaker 122, which has been included in the input audio picked upvia acoustic coupling between the loudspeaker 122 and the microphone 74.When functioning properly, the echo canceller 115 can reduce the chancesthat the far-end 30 will detect an “echo.”

In most instances, the analog-to-digital (A/D) clock 68 for the portabledevice's microphone 74 is not matched in frequency to thedigital-to-analog (D/A) clock 118 of unit's loudspeaker 122. Themismatched clocks 68 and 118 can result in poor performance of theacoustic echo cancellation. For proper cancellation, for example, theloudspeaker's D/A clock 118 and the microphone's A/D clock 74 need to beat the same frequency within a small variation (e.g., less than 1 partsper million (PPM)). Being on separate devices, however, theloudspeaker's and microphone's clocks 68 and 118 are controlled byphysically separate crystals, and their frequencies may be off by 100PPM or more.

G. Clock Compensation

To deal with the mismatched clocks 68 and 118, the system 10 uses aclock compensator 160 at the unit 100 and/or a clock compensator 170 atthe portable device 50 so the echo canceller 115 on the unit 100 canmore effectively remove echo caused by acoustic coupling between theunit's loudspeaker 122 and the device's microphone 74. To do this, theclock compensator 160, 170 compensates for the clock differences andimproves the performance of the unit's echo canceller 115.

In one embodiment, the clock compensator 160 on the videoconferencingunit 100 assists in the echo cancellation. In particular, the clockcompensator 160 cross-correlates an echo estimate of the far-end audio(being output by the unit's loudspeaker 122) and the near-end audio(being input from the device's microphone 74). Using thecross-correlated results, the compensator 160 then adjusts a sample rateconversion factor used for the echo cancellation analysis of the audioto be sent to the far-end 30. In this arrangement, the clock compensator160 using the echo estimate can compensate for the differences in theclocks 68 and 118 without involving the portable device 50.

In another embodiment, the clock compensator 170 on the portable device50 assists in the echo cancellation by determining a difference in theclocks 68 and 118. The difference is then used to “synchronize” theaudio clocks 68 and 118 by adjusting the sample conversion rate of theportable device's microphone 74 or by adjusting the sample rateconversion for the unit's echo canceller 115 or the unit's loudspeaker122.

As noted above, the portable device 50 pairs with the unit 100 byobtaining the unit's IP address in an acoustic or ultrasonic beaconemitted by the unit's loudspeaker 122. This ultrasonic beacon ispreferably transmitted with a carrier signal with an ultrasonicfrequency of 21 kHz, for example. After capturing the beacon, theportable device 50 uses the frequency of the ultrasonic carrier signalto determine the frequency difference between the clock 68 for itsmicrophone 74 and the clock 118 for the unit's loudspeaker 122.

For example, the unit 100 emits the acoustic or ultrasonic beacon asnoted above at a pre-determined frequency at 21,000 Hz using theloudspeaker 122. The microphone 74 on the portable device 50 capturesthe ultrasonic beacon, and the compensator 170 measures the beacon'sfrequency relative to the unit's A/D clock 68. In general, the measuredfrequency will differ from the beacon's pre-determined frequency (21kHz) due to the different frequencies of the separate clocks 68 and 118.By way of example, the microphone's clock 68 may be about 1 percentslower than the loudspeaker's clock 118. In this case, the compensator170 will measure the frequency of the beacon as 21,210 Hz (i.e.,21,000×1.01).

Despite this measured difference, the portable device 50 with themicrophone 74 and conferencing application (66) knows that the actualfrequency of the beacon should be the predetermined 21,000 Hz.Therefore, the compensator 170 can deduce that the clock frequencydifference is 210 Hz (i.e., 21,210 Hz−21,000 Hz). Once the frequencydifference is known to within an acceptable deviation (e.g., 1 PPM), theportable device 50 can use re-sampling algorithms and can adjust themicrophone's sample rate conversion to match the loudspeaker's samplerate conversion. In this way, the A/D converter 69 based on the device'sclock 68 can having its sample conversion rate match that of the D/Aconverter 119 based on the unit's clock 118. Once the rates match, theecho canceller 115 on the unit 100 can operate more effectively.

The reverse is also possible where loudspeaker's sample rate conversionis adjusted. For example, the portable device 50 can send the determinedfrequency difference to the unit 100 via the network connection. Inturn, the compensator 160 of the unit 100 can use re-sampling algorithmsand can change the loudspeaker's sample rate conversion used foroutputting audio so it will match the microphone's sample rateconversion. Alternatively, the compensator 160 can use this determineddifference in the sample rate conversion of the echo cancellation so theecho canceller 115 can operate more effectively.

In a further alternative, the portable device 50 may not actuallycalculate the frequency difference. Instead, the portable devicecommunicates its audio capture frequency of its microphone 74 respondingto the acoustic or ultrasonic beacon to the videoconferencing unit 100using the network connection. Then, the videoconferencing unit 100 cancalculate the frequency difference between the predetermined carriersignal frequency and the audio capture frequency communicated to it fromthe portable device 50. Thus, the procedures discussed above would applyto the unit's processing instead of the device's processing.

In the current example, only the audio between one portable device 50and the unit 100 has been discussed. In general, however, there can beany number of devices 50 in the room, and each can have its own A/Dconverter 69 and clock 68 for its microphone 74. These devices 50 cansend their digital audio streams to the videoconference unit 110 via thewireless connection, and some of these streams may contain output audiofrom the loudspeaker 122 that has been acoustically coupled to thedevices' microphone 74. The teachings above can be applied to the audioof each of the portable devices.

The foregoing description of preferred and other embodiments is notintended to limit or restrict the scope or applicability of theinventive concepts conceived of by the Applicants. It will beappreciated with the benefit of the present disclosure that featuresdescribed above in accordance with any embodiment of aspect of thedisclosed subject matter can be utilized, either alone or incombination, with any other described feature, in any other embodimentor aspect of the disclosed subject matter.

In exchange for disclosing the inventive concepts contained herein, theApplicants desire all patent rights afforded by the appended claims.Therefore, it is intended that the appended claims include allmodifications and alterations to the full extent that they come withinthe scope of the following claims or the equivalents thereof.

What is claimed is:
 1. A videoconferencing method, comprising:connecting a portable device in a videoconference connection to afar-end; outputting at the portable device far-end video and far-endaudio received from the far-end via the videoconference connection;sending near-end video and near-end audio obtained with the portabledevice to the far-end via the videoconference connection; connecting theportable device in a wireless connection to a videoconferencing unit;and transferring the videoconference connection from the portable deviceto the videoconferencing unit.
 2. The method of claim 1, wherein thevideoconference connection comprises an Ethernet connection, a Wide AreaNetwork (WAN) connection, a wireless connection, an Internet connection,a cellular connection, a Plain Old Telephone Service (POTS) connection,or a combination thereof.
 3. The method of claim 1, wherein the wirelessconnection comprises a Local Area Network (LAN) connection, an intranetconnection, a Wireless Local Area Network (WLAN) connection, a PersonalArea Network (PAN) connection, a Wireless Fidelity (Wi-Fi) connection, awireless connection, or a combination thereof.
 4. The method of claim 1,wherein transferring comprises switching output of at least one of thefar-end video and the near-end audio from the portable device to thevideoconferencing unit.
 5. The method of claim 4, wherein switchingoutput of at least one of the far-end video and the near-end audio fromthe portable device to the videoconferencing unit comprises: receivingthe far-end video and the near-end audio from the far-end at thevideoconferencing unit; outputting the far-end video to a displayoperatively coupled to the videoconferencing unit; and outputting thefar-end audio to a loudspeaker operatively coupled to thevideoconferencing unit.
 6. The method of claim 1, wherein transferringcomprises switching obtaining of at least of the near-end video and thenear-end audio from the portable device to the videoconferencing unit.7. The method of claim 6, wherein switching obtaining of at least of thenear-end video and the near-end audio from the portable device to thevideoconferencing unit comprises: obtaining the near-end audio with amicrophone of the portable device; and sending the near-end audio viathe wireless connection from the portable device to thevideoconferencing unit for sending to the far-end.
 8. The method ofclaim 7, further comprising cancelling the far-end audio output by aloudspeaker of the videoconference unit and present in the near-endaudio obtained with the microphone of the portable device.
 9. The methodof claim 8, further comprising compensating for a frequency differencebetween a first clock of an analog-to-digital converter for themicrophone of the portable device and a second clock of adigital-to-analog converter for the loudspeaker of the videoconferencingunit.
 10. The method of claim 1, further comprising controlling afunction of the videoconferencing unit with an application operating onthe portable device.
 11. The method of claim 1, further comprisingsending content residing at the portable device to the videoconferenceunit via the wireless connection for incorporation into thevideoconference.
 12. A programmable storage device having programinstructions stored thereon for causing a programmable control device toperform a method of claim
 1. 13. A portable device, comprising: aplurality of interfaces communicatively connecting to avideoconferencing unit, a camera, a loudspeaker, a display, amicrophone, and a far-end; and a processing unit communicativelyconnected to the interfaces and configured to: connect in avideoconference connection to the far-end; output far-end video with thedisplay and far-end audio with the loudspeaker; send near-end videoobtained with the camera and near-end audio obtained with the microphoneto the far-end via the videoconference connection; connect in a wirelessconnection to the videoconferencing unit; and transfer thevideoconference connection from the portable device to thevideoconferencing unit.
 14. A videoconferencing method, comprising:connecting a videoconferencing unit in a videoconference connection to afar-end; connecting the videoconferencing unit in a wireless connectionto a portable device in an environment of the videoconferencing unit;obtaining first audio captured with the videoconferencing unit in theenvironment; sending the first audio to the far-end via thevideoconference connection; obtaining, at the videoconferencing unit viathe wireless connection, second audio captured with the portable devicein the environment; and switching to sending the second audio to thefar-end via the videoconference connection.
 15. The method of claim 14,wherein the videoconference connection comprises an Ethernet connection,a Wide Area Network (WAN) connection, a wireless connection, an Internetconnection, a cellular connection, a Plain Old Telephone Service (POTS)connection, or a combination thereof; and wherein the wirelessconnection comprises a Local Area Network (LAN) connection, an intranetconnection, a Wireless Local Area Network (WLAN) connection, a PersonalArea Network (PAN) connection, a Wireless Fidelity (Wi-Fi) connection, awireless connection, or a combination thereof.
 16. The method of claim14, wherein the second audio captured with the portable device in theenvironment comprises audio captured with a microphone of the portabledevice.
 17. The method of claim 14, wherein switching comprises sendingthe second audio when having a greater level than the first audio. 18.The method of claim 14, further comprising: obtaining first videocaptured with the videoconferencing unit; sending the first video to thefar-end via the videoconference connection; obtaining, at the unit viathe wireless connection, second video captured with the portable devicein the environment; and switching to sending the second video to thefar-end via the videoconference connection.
 19. The method of claim 18,wherein the second video captured with the portable device in theenvironment comprises video captured with a camera of the portabledevice.
 20. The method of claim 18, wherein switching comprises sendingthe second video when the second audio has a greater level than thefirst audio.
 21. The method of claim 14, further comprising: obtaining,at the videoconferencing unit via the wireless connection, contentresiding at the portable device; and incorporating the obtained contentinto the videoconference.
 22. A programmable storage device havingprogram instructions stored thereon for causing a programmable controldevice to perform a method of claim
 14. 23. A videoconferencingapparatus, comprising: a microphone for capturing audio of theenvironment; one or more interfaces for communicatively connecting to aportable device and a far-end; and a processing unit communicativelyconnected to the camera, the display, the microphone, the loudspeaker,and the one or more interfaces, the processing unit configured to:connect in a videoconference connection to the far-end; connect in awireless connection to the portable device in the environment; obtainfirst audio captured with the microphone; send the first audio to thefar-end via the videoconference connection; obtain, at the unit via thewireless connection, second audio captured with the portable device; andswitch to sending the second audio to the far-end via thevideoconference connection.